Reinforcement Learning for Rate-Distortion Optimized Hierarchical Prediction Structure

Video coding standards use a prediction structure to arrange video frames and exploit temporal correlations. In this aspect, it is crucial resolve complicated dependencies among improve efficiency because the of preceding frame affects rate-distortion (R-D) performance subsequent frames. Previous algorithms have attempted address problem using handcrafted features or analytical models even though natural videos display various characteristics. paper, we propose reinforcement learning (RL)-based decision algorithm build optimal hierarchical under random-access configuration (RA-HPS) in Versatile Coding (VVC). Our goal maximize by selecting series group pictures (GOP) structures for coding. Accordingly, formulate an adaptive GOP selection with binary tree represent policy. We generate minimize sum R-D costs all plausible trees. A new RL policy representation defined, obtained sequential update. The grows state-action reward sequence each node. For efficient learning, proposed technique uses deep Q-network architecture capture correlation between frames, which helps learn tree-based framework effectively. Experimental results demonstrate that achieves significant Bjontegaard-Delta (BD)-rate reduction compared state-of-the-art size-selection algorithms.